<html>
<head><meta charset="utf-8"><title>general · t-lang/wg-unsafe-code-guidelines · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/index.html">t-lang/wg-unsafe-code-guidelines</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general.html">general</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="133076899"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general/near/133076899" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jake Goulding <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general.html#133076899">(Aug 30 2018 at 18:21)</a>:</h4>
<p>That'd be <code>#[repr(frank)]</code></p>



<a name="133076935"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general/near/133076935" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jake Goulding <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general.html#133076935">(Aug 30 2018 at 18:22)</a>:</h4>
<p>(Since the meeting is over...)</p>



<a name="133076963"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general/near/133076963" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general.html#133076963">(Aug 30 2018 at 18:22)</a>:</h4>
<p>^^</p>



<a name="133077005"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general/near/133077005" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Frank McSherry <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general.html#133077005">(Aug 30 2018 at 18:23)</a>:</h4>
<p>I did try and flip through them and see if one was "the obvious fit", but I wasn't entirely sure. The abom stuff (as well as a few related things, like Rust-on-Rust FFI) don't exactly care <em>what</em> the layout is, just that it stays put. So, it may be weaker than needing a defined binary repr.</p>



<a name="133077061"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general/near/133077061" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Frank McSherry <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general.html#133077061">(Aug 30 2018 at 18:24)</a>:</h4>
<p>But, if there is an obvious place to watch (e.g. data-invariants) or just for the next meeting, I'll do that and get back to my book. :)</p>



<a name="133077109"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general/near/133077109" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general.html#133077109">(Aug 30 2018 at 18:25)</a>:</h4>
<p><span class="user-mention" data-user-id="116609">@Frank McSherry</span> yeah I think what you are asking about is mostly happening several layers higher... like, safety invariants for references permitting transmute of <code>&amp; &amp;mut T</code> to <code>&amp;&amp;T</code> or so</p>



<a name="133077179"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general/near/133077179" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general.html#133077179">(Aug 30 2018 at 18:26)</a>:</h4>
<p>I was thinking the same thing; this <a href="https://github.com/rust-rfcs/unsafe-code-guidelines/issues/8" target="_blank" title="https://github.com/rust-rfcs/unsafe-code-guidelines/issues/8">proposed topic of validity invariants</a> might be relevant, but we're not there yet</p>



<a name="133077194"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general/near/133077194" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general.html#133077194">(Aug 30 2018 at 18:26)</a>:</h4>
<p>but if that gives you any hope, the formal model we built last year to prove Rust's safety was specifically taking into account <a href="http://www.frankmcsherry.org/serialization/2015/05/04/unsafe-at-any-speed.html" target="_blank" title="http://www.frankmcsherry.org/serialization/2015/05/04/unsafe-at-any-speed.html">http://www.frankmcsherry.org/serialization/2015/05/04/unsafe-at-any-speed.html</a> so that we could prove a transmute from <code>&amp;&amp;T</code> to <code>&amp;Box&lt;T&gt;</code>^^</p>



<a name="133077199"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general/near/133077199" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general.html#133077199">(Aug 30 2018 at 18:26)</a>:</h4>
<p>it seems like the biggest area of concern would be exactly those places where the "definition of memory" deviates from "just bytes" — e.g., references and (maybe but probably not?) uninit memory</p>



<a name="133077288"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general/near/133077288" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Frank McSherry <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general.html#133077288">(Aug 30 2018 at 18:28)</a>:</h4>
<p>Yeah, I recall you observing that the "hard part" of abomonation looked plausible. I'm mostly just planning on showing up so that no one forgets that something like it is useful too. I.e., in the glorious battle between "zomg random data layout opts" and "zomg please stop we are trying to make computers fast", I represent the latter.</p>



<a name="133077320"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general/near/133077320" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jake Goulding <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general.html#133077320">(Aug 30 2018 at 18:29)</a>:</h4>
<p>I think both teams are on the "make go faster" side ;-)</p>



<a name="133077416"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general/near/133077416" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Frank McSherry <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general.html#133077416">(Aug 30 2018 at 18:30)</a>:</h4>
<p><span class="user-mention" data-user-id="120791">@RalfJ</span> I could totally update that post, if it would help (stroke-be-less-harmful); there are several things that are wrong in it (you've probably noticed, but lots of other people noticed too; I opted not to edit it once it was live).</p>



<a name="133077506"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general/near/133077506" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Frank McSherry <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general.html#133077506">(Aug 30 2018 at 18:31)</a>:</h4>
<p><span class="user-mention" data-user-id="116155">@Jake Goulding</span> yes, sorry that's totally fair. Maybe the real point is that optimizing/randomizing layouts come with a cost, and I am here to represent that (vs just hearing about the great advantages of layout re-optimization).</p>



<a name="133077625"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general/near/133077625" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general.html#133077625">(Aug 30 2018 at 18:33)</a>:</h4>
<p><span class="user-mention" data-user-id="116609">@Frank McSherry</span> whatever you prefer, I think I got out of that post what I can.^^ that single transmute is, to me, the essence of what is going on. (the post does it for <code>Vec</code>, of course, but that makes no fundamental difference.)</p>



<a name="133077733"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general/near/133077733" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jake Goulding <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general.html#133077733">(Aug 30 2018 at 18:35)</a>:</h4>
<p>I think everyone knew what you meant, so it wasn't a slam.</p>
<p>It's most likely all about control and visibility...</p>
<p>I'd guess that your POV is that you know more beyond what the compiler does/can so you want the ability to control the layout to increase speed. <br>
I'd guess the compilers POV is that it knows more beyond what the user does/can so it wants the ability to control layout to increase speed.</p>



<a name="133077816"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general/near/133077816" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Frank McSherry <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general.html#133077816">(Aug 30 2018 at 18:36)</a>:</h4>
<p><span class="user-mention" data-user-id="120791">@RalfJ</span>  I think I'll get to re-write it soon. Timely just got a zero-copy dataplane, and part of that is "not copying from bytes to typed data" (courtesy abom) and I suspect explaining that will require re-explaining abom. Btw, have you seen / have opinions on <a href="https://github.com/frankmcsherry/blog/blob/master/posts/2017-07-27.md" target="_blank" title="https://github.com/frankmcsherry/blog/blob/master/posts/2017-07-27.md">https://github.com/frankmcsherry/blog/blob/master/posts/2017-07-27.md</a>, in which I pretend that abomonation is a lot like region allocation?</p>



<a name="133077903"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general/near/133077903" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Frank McSherry <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general.html#133077903">(Aug 30 2018 at 18:38)</a>:</h4>
<p><span class="user-mention" data-user-id="116155">@Jake Goulding</span> &gt; I'd guess that your POV is that you know more beyond what the compiler does/can so you want the ability to control the layout to increase speed.</p>
<p>It's a bit weaker than this, personally. I'm up for the compiler determining the layout, but I need some structure to know when I can re-interpret bytes. If compiler says "they must be like this" I'm good with that as long as it stays true for a while. At the moment, I'm weirded out by the fact that the compiler can change this with every recompile, and as I understand it doesn't directly promise this (but is boxed in by needing to be able to link dynamically with other libraries).</p>



<a name="133078367"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general/near/133078367" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jake Goulding <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general.html#133078367">(Aug 30 2018 at 18:46)</a>:</h4>
<p>Ah, I assumed that you were constrained by a pre-existing external layout (yon blobs of bytes on the disk)</p>



<a name="133078605"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general/near/133078605" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Frank McSherry <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general.html#133078605">(Aug 30 2018 at 18:51)</a>:</h4>
<p>Most of my uses are "I have a program, it wants to turn types to bytes and back to &amp;types". It would also be cool to have a stable layout for longer term storage, but I think at that point you want the grown-up Protobuf, CapnProto, FlatBuffers stuff (or <em>shudder</em> JSON).</p>



<a name="133078693"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general/near/133078693" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Frank McSherry <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general.html#133078693">(Aug 30 2018 at 18:53)</a>:</h4>
<p>As one example, that I think is pretty hard with lots of approaches: it would be great to be able to memmap a blob of data and have it behave as a &amp;type, allowing random access without needing a full scan of the data. The systems-level difference between "process starts; data read immediately, go" vs "process starts, rescans 100GB of data, go" is huge in the space I'm working in (big data, fault-tolerance, blah blah).</p>



<a name="133078858"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general/near/133078858" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jake Goulding <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general.html#133078858">(Aug 30 2018 at 18:56)</a>:</h4>
<p>I was saddened to realize that <code>mmap</code> will always be <code>unsafe</code> because you cannot guarantee that you are the sole owner of a piece of memory (a.k.a. a big point of <code>mmap</code>)</p>



<a name="133080783"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general/near/133080783" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general.html#133080783">(Aug 30 2018 at 19:31)</a>:</h4>
<p>Also I feel I have to make a declaration: While <span class="user-mention" data-user-id="116609">@Frank McSherry</span> and <span class="user-mention" data-user-id="116155">@Jake Goulding</span> are in different camps of the "make computers be fast" team (I know you're not really but bear with me here ;), I am squarely in the "make computers be sane" team. I know I am fighting windmills, but I will be a bastion of sanity against everyone who intents to sacrifice everything and their grandma on the altar of performance. Too long have the evangelists of speed reigned in the space of compilers, it is time we break their rule and make programming languages beautiful again!</p>



<a name="133080862"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general/near/133080862" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jake Goulding <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general.html#133080862">(Aug 30 2018 at 19:32)</a>:</h4>
<p><span class="user-mention" data-user-id="120791">@RalfJ</span> I claim no camp.</p>



<a name="133080883"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general/near/133080883" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general.html#133080883">(Aug 30 2018 at 19:32)</a>:</h4>
<p><span class="user-mention" data-user-id="116155">@Jake Goulding</span> I know but the story worked better this way :P</p>



<a name="133080899"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general/near/133080899" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jake Goulding <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general.html#133080899">(Aug 30 2018 at 19:32)</a>:</h4>
<p>Do I get to be the Jedi or the Sith</p>



<a name="133080929"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general/near/133080929" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general.html#133080929">(Aug 30 2018 at 19:33)</a>:</h4>
<p>I leave that up to you. as far as I am concerned, performance people are Sith and of course I am on the light side. ;)</p>



<a name="133081959"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general/near/133081959" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general.html#133081959">(Aug 30 2018 at 19:51)</a>:</h4>
<p>wow, I had no idea you had such poetry in you, <span class="user-mention" data-user-id="120791">@RalfJ</span></p>



<a name="133082264"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general/near/133082264" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general.html#133082264">(Aug 30 2018 at 19:56)</a>:</h4>
<p>Still new to this, particularly in English. ;)</p>



<a name="133087387"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general/near/133087387" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Nicole Mazzuca <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general.html#133087387">(Aug 30 2018 at 21:25)</a>:</h4>
<p>I personally think computers are way too fast; we should switch back to before all this caching BS :3</p>



<a name="133411641"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general/near/133411641" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> alercah <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/general.html#133411641">(Sep 06 2018 at 00:06)</a>:</h4>
<p><span class="user-mention" data-user-id="120791">@RalfJ</span> I think that, a year ago, people might have melted down at the suggestion that speed was not the one true god. But with the spectre of 2018 looming close still, I think that people may be willing to, at the very least, speculate alternatives, and so I'll support you in this execution.</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>